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ABSTRACT 


Strong  consistency  and  asymptotic  normality  of  an  estimator  re- 
lated to  least  squares  estimator  for  parameters  involved  in  nonlinear 
stochastic  differential  equations  are  investigated  by  studying  families 
of  stochastic  Integrals  using  Fourier  analytic  methods. 


AMS  (1980)  Subject  Classification:  Primary  62M05,  Secondary  60H10 
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1.  Introduction 


Recently  there  is  a growing  interest  in  the  study  of  inference 
problems  for  stochastic  processes  both  continuous  and  discrete  time 
in  view  of  the  large  number  of  applications  to  engineering  problems. 

It  has  been  found  that  the  class  of  diffusion  processes  is  amenable 
for  statistical  analysis.  A survey  of  the  recent  work  in  this  area 
is  given  in  Basawa  and  Prakasa  Rao  (1979).  Further  work  on  asymptotic 
theory  of  maximum  likelihood  and  Bayes  estimators  for  parameters  of 
diffusion  processes  is  discussed  in  Prakasa  Rao  (1979a). 

Dorogovchev  (1976)  studied  weak  consistency  of  least  square 
estimators  for  parameters  of  diffusion  processes  which  are  solutions 
of  non-linear  stochastic  differential  equations.  Asymptotic  normality 
and  asymptotic  effiency  of  these  estimators  is  investigated  in  Prakasa 
Rao  (1979b).  Our  aim  in  this  paper  is  to  study  limiting  properties  of 
a process  related  to  least  squares  estimator  and  hence  to  discuss  the 
asymptotic  properties  of  an  estimator  derived  from  the  limiting  process. 
We  study  strong  consistency  and  asymptotic  normalityof  this  estimator. 
Our  approach  here  is  entirely  different  from  that  of  Dorogovchev  (1976) 
and  Prakasa  Rao  (1979b).  We  believe  that  our  techniques  for  study  of 
families  of  stochastic  integrals  is  new  and  is  of  independent  Interest. 


[ 
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2.  Study  of  process  related  to  least  squares  estimator 

Let  { X ( t ) , t _>  0)  be  a real-valued  stationary  ergodic  process 
satisfying  the  stochastic  differential  equation 

dX(t)  = f(e0,X(t))dt  + de(t),  X (0 ) * XQ,  t>0 

2 2 

where  £( t)  is  a Wiener  process  with  mean  zero  and  variance  at,  a 

p 

being  unknown  and  E[XQ]  < ■».  Suppose  f(e,x)  is  a known  real-valued 
function  continuous  on  © x R where  © is  a closed  interval  on  the  real 
line  and  eQ € © is  unknown.  Without  loss  of  generality,  assume  that 
© = [-1,1]. 

Suppose  the  process  (X(t),  0 <_  t <_  TJ  is  observed  at  time  points 
t^,  k = 0,1,..., n-1  with  tQ  = 0 and  tn  * T.  Let 

T "l1  CX(tfc+1)  - X(t.)  - f(6,X(tJ)AtJ2 

qT(9)  . i M k k k_  . 

n k=0  \ 

where  Atk  = t^-t^,  e 1 k <_  n-1 . An  estimator  en  y which  minimizes 

qT(o)  over  0€G  is  called  a least  squares  estimator  of  0.  Assume  that 

a T 

such  an  estimator  exists.  Note  that  if  0 T minimizes  Q_(0),  then  it 

n , i n 

minimizes  qj(o)  - Q„(e0) - 

We  shall  first  study  the  limiting  properties  of  the  process 
1 Qp ( () ) - Q^(0g),  oe©}as  the  norm  of  division  Ap  = max  | | tends 

to  zero.  Let  AXk  = X(tR+1 ) - X(tk)  and  A£k  = £(tk+1)  - C(tk),0  < k < n-1 
For  any  fixed  0, 
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Q„T(e)-q;(90) 


l 


Atk  tAXk-f(M(tk»atk] 


" H 4tk  ^lXk'f*VX^tk’*itk^ 


, k+1 

= I {J  f(e0,x(t))dt+Ack-f(e,x(tk))Atk}‘ 
k k tk_] 


uk+l 


l {l"  ' f(e0*x(t))dt+^k-f(e0,x(tk))Atk}2 

k k tk 


k+1 


1 ^ 9 

• l t=-  {/  Cf(en,x(t))-f(e,X(t.  ))]dt+Aclr) 


k Atk  t 


k+1 


1 K-r  | a 

- I -5-  {/  [f(e0,X(t))-f(e0,X(tk))]dt+A£k}'  . 

k k tk 


It  Is  easy  to  check  that 
(2.0)  Q^(e )"Qn(e0) 


- j>  [f(e0.X(tk))-f(e,X(tk))]2Atk 


+ 2 l [f(e0,X(tk)-f(e,X(tk)]Ack 


k+1 


+ 2 y (f(0o,X(tk)-f(e,X(tk))}  / {f(e0.X(t))-f(90,X(tk))}dt 

k tk 


Mn  + 2I2n  + 2I3n 


Assume  that  the  regularity  condition  on  f(x,e)  stated  at  the  end  of 
this  section  are  satisfied.  Since  f(e,x)  Is  continuous  In  x and  the 


; 


process  X has  continuous  sample  paths  with  probability  one.  It  follows 


that 

(2-D  Iln  a^'  / [f(e0,X(t))-f(e,X(t))]2dt 

0 

as  An  -+  0.  Assumption  (A2)  implies  that 

(2.2)  I2n  / [f(eo,X(t))-f(0,X(t))jde(t) 

as  &n  ->  0 in  view  of  stationarity  of  the  process  X where  the  last  in- 
tegral is  the  Ito-stochastic  integral. 

Let  us  now  estimate  I3f).  In  view  of  assumption  (A4),  it  can  be 
checked  that 
tk+l 

(2.3)  |/  {f(e0,X(t))-f(60,X(tk)))dt| 

tk 


k+1 

<L(e0)/  |X(t)-X(tk)|dt 


tk+l  t 

1 L(eQ)/  (U(t)-c(tk)|+/  f(e0,X(s))|ds}dt 


tk+l  , tk+l  t 

< L(e0)/  | c(t)-c(t.  ) jdt+L2(e0)/  [ /{l+|X(s)(}ds]  dt 

tL  tL  tL 


< L(e0)Atk  sup  U(t)-c(tk)|+L2(e0)Atk 

tk-t-tk+l 

< L(eQ)Atk  sup  U(t)-?(tk)|+L2(0o)At2 

tk-t-tk+l 


sup  / {l+|X(s)j }ds. 


tk-t-tk+l  fck 


sup  (1+1 X(t) | } 

‘k^Vt 


for  0 < k _<  n-1 . Using  assumption  (A4)  again,  we  obtain  the  following 
inequality: 


<2-4)  !3n  1 I J(x(tk))|L(0o)*Atk  sup  U(t)-c(t.)| 

k * t <ti<t  * 

K— — k+1 

* L2(e0)at^  sup  (l+|X(t)|)l|8-e„|. 

1 

Since  J(-)  is  continuous  and  X(*)  has  continuous  sample  paths  al- 
most surely,  it  follows  that  there  exists  a constant  C*(0q)  depending 
on  T only  such  that 

(2.S)  I3n  < c*(e0){IAV  sup  UCt)-c(tk)|  + I*tj;)|e-e0|  . 

lk  tkit£tk+1  k » 

Since  ee  6 compact,  it  follows  that 

!3n  i c(0O){£Atk(1+Atk)(2Atk  l09  VAtk)1/2  + At2}  a.s. 

whenever  ar  is  sufficiently  small  by  the  law  of  iterated  logarithm  for 
Brownian  increments  (cf.  McKean  (1969),  p.14).  Therefore 

(2-6)  J3n  - 0([  Atk/2  log  1/2  1/Atk)  a.s. 

uniformly  in  efc  6 . Furthermore  the  convergence  in  (2.1)  is  uniform  in 
0 € 0 since 

|f(e0,X(t))-f(e,x(t))|2  < |0o-e|2j2(x(t))  < c J2(x(t)) 

and  J(X(t))  is  integrate  pathwise  on  [0,T]  by  (A4).  Here  we  have  used 
the  fact  that  6 is  compact.  Hence 

T . 

(2-7)  Iln  * / [f(eo.X(t)-f(0,X(t)3  dt  + o(l)  a.s. 

uniformly  In  e as  An  -*•  0.  We  shall  discuss  uniform  convergence  of  l£n 
In  the  next  section. 

Relations  (2.0),  (2.6)  and  (2.7)  show  that,  for  any  fixed  T, 
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(2.8)  Q>)-Q>0)  = / [f(0o,X(t))-f(e,X(t))]2dt  + I2n+o(l)  a.s. 

uniformly  in  et  b compact  as  An  -*■  0 where  I^n  satisfies  relation  (2.2). 
Let  us  consider  the  limiting  process 

T ? 

(2.9)  RT(e)  = / [f(e0,X(t))  - f(e,X(t))]2dt 

T 

+ 2 / [f(en,X(t))  - f(e,X(t))]dc(t) 
o u 

T ? T 

= / /(0,X(t))dt  - 2 / v(e,x(t))de(t) 

0 0 

where 

(2.10)  v(e,x)  = f(e,x)  - f(eQ,x). 

We  study  the  limiting  properties  of  the  process  (RT(e),  0€0}in 
the  next  section. 

Assumptions 

(A1 ) f(e,x)  is  continuous  in  (e,x)  and  differentiable  with  respect  to 
o.  Denote  the  first  partial  derivative  of  f with  respect  to  0 by 
f^(o.x)  and  the  derivative  evaluated  at  0Q  by  f^(©0,x). 

(A2)  ECfJ^feQ.XtO))]2  < - 

(A3)  f^(e,x)  is  Lipschitzian  in  0 for  each  x i.e.,  there  exists  a > 0 

such  that 

I fg1  ^(e.x)-fg1  ^U,x)|  < c(x)|0-$|a  , xtR,0,|f e 
and 

E[c2(X(0))]  < ». 

(A4)  f(o,x)  satisfies  the  following  conditions: 
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(i)  |f(e,x)|  < L(e)(l+|x| ) , e e g , x e R ; sup{L(e):  ot  ©}  <*. 

(ii)  |f(0,x)-f(e,y)|  < L(0 ) I x-y  I , e€G,x,y,tR. 

(ill)  I f(e»x)-f(<j.,x)|  <_  J(x) 1 0-4» | . 0 , e , x € R 

where  J(*)  is  continuous  and  E[J2(X(0))]  < <=°. 

(A5)  1(0)  = E[f(0,X(O))-f(eo,X(O)]2  >0  for  0 f o . 

2 

Remark;  Since  E[X  (0)]  < <*>,  assumption  A4(i)  implies  that 

E[f(6,X(0))]2  < ® 

for  all  9€  G . 

3.  Study  of  a limiting  process  related  to  least  squares  estimator 

Let  us  now  study  the  properties  of  the  limiting  process 

, T 

(3-D  ZT(e)H-Jr  / v(e,X(t))dt(t) 

1 /r  o 

as  a process  in  the  parameter  0€  © = [-1,1]  as  T + From  the  central 
limit  theorem  for  stochastic  integrals  (cf.  Basawa  and  Prakasa  Rao  (1979)), 
it  can  be  shown  that 

~ / v(e,X(t))d?(t)  N(O,E[v(0,X(O))]2o2) 

/f  0 

since  the  process  X is  stationary  ergodic.  In  general,  finite  dimension- 
al distributions  of  the  process  {ZT(0),  e€©}  converge  to  the  finite 
dimensional  distributions  of  the  Gaussian  process  { Z( e ) , e 6 ©}  with  mean 
zero  and  covariance  function 

R(01 ,02)  = E[v(01,X(O))v(02,X(O))]a2. 

We  shall  now  prove  the  weak  convergence  of  the  process  { Z^. (e),  0€G} 

on  C[-l,l]  under  uniform  norm.  It  is  sufficient  to  prove  that 

(3.2)  lim  llm  P(  sup  | ZT(0)-ZT(<j>)  | > c)  = 0. 

T-*»  6->0  | 0-<£  ( <6 

i 

1 
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Since  v(o,x)  is  differentiable  with  respect  to  0 on  [-1,1]  by 
assumption  (Al),it  is  easy  to  see  that  there  exists  a cubic  polynomial 


g(o,x)  in  e such  that 


g(-l,x)  = v(  l,x),  g(l,x)  = v(l,x) 


. -0) 


9l  w(-l,x)  = v'  ' (-1 ,x) , gy;(l,x)  = v^'O.x). 


0)n  = „0)| 


h(e,x)  = v(e,x)-g(e,x). 


Then  h(-l,x)  = h(l,x)  = 0,h^H-l,x)  *h^\l,x)  = 0.  Now 


i T i T 

(3.3)  Z (e)=-1  / h(e,x(t))d5(t)  + j-  / g(e,x(t))dc(t). 
1 /f  0 /To 


Since  g(e,x)  is  a cubic  polynomial  in  8 with  coefficients  in  x which  are 
linear  functions  of  v(-l,x),  v(l  ,x)  ,v^(-l  ,x)  andv^\l,x),  it  is  easy 


to  check  the  uniform  equi-continuity  condition  of  type  (3.2)  for 


1 T 

— / g(e,X(t))dc(t). 
/T  0 


Let  us  now  consider  the  process 


(3.4) 


WT(0)  = ~ / h(0,X(t))dc(t). 
1 A 0 


Let  the  Fourier  expansion  for  h(0,x)  in  L2([-l,l])  be  given  by 


(3.5) 


h(o,x)  = l an(x)eT 
n 


Lemma  3.1 


T T 

(3.6)  / h(0,X(t))dc(t)  = l { / a (X(t))dc(t)}eT 

0 n 0 " 


in  the  sense  of  convergence  in  quadratic  mean. 
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Proof  An  approximating  sum  in  L^-norm  for 


/ h(e,X(t))d£(t) 
0 


is 


'in  ■ L 

J * 


and  an  approximating  sum  in  L9-norm  for  £ {/  a (X(t))dc(t)}en1n0  is 

c n 0 


i «"n6(  i >„(*(t, ,)«,) 

|nf<H  j=l  " J’1  J 


N 


2NM 


It  is  sufficient  to  prove  that  E|A1N-A?NM|  ->  0 as  N -+  °°  and  M ->  ». 
2 


'IN  2NM 

N M 


ElfllH-A2NMi  * E|  I C(Me.x(t I 1 

j = I n=-M 

- E|  l l a ( X ( t • -j ) )eirinoAC  • 
j-1  1 n ] >M  " 31  3 


e-nean(X(tj.1))}AS.|2 


irine.^  1 2 


< [ I (E  (I  a (X(t . -j  ))A4i)  }^] 
~ | n | >M  J-1  n J 

by  the  elementary  inequality 


Ell  Vnl2i<^inl(E(yn»4>2 

n n 


for  any  sequence  of  complex  numbers  {xn>  and  any  sequence  of  real 


valued  random  variables  (Yn>  n >_  1).  Hence 


eia1n-a2hhI2  ^ c|nf>H ( jX,  E<a„<x<Vi,)2AVi]2  • 


Since 


N O T o 

l E(a  (X(t . JTAt.  + / E{a  (X(t)Tdt  = Tun  (say), 

j_1  " J-<  J o 


Now 


remarks  following  Lemma  3 of  the  appendix  under  assumption  (A3). 
Let 


<3-7)  W = -1  / a (X(t))dC(t). 

n /To 

Lemma  3.2.  For  every  e > 0, 

(3-8)  lim  P(  sup  | WT(e )-WT(<j> ) | > e)  = 0 

6-*0  ( e-<|)  ( <6 

for  every  T > 0. 

Proof.  In  view  of  Lemma  3.1,  for  any  e > 0, 

(3.9)  P(  sup  |WT(e)-WTU)|  > e) 

|e-(j>|<6  1 


= p( 

sup 

CO 

1 1 

w (eirine-eirin*)|  > e) 

|e-<f>|<6 

n=-® 

n 1 ' 

< p( 

sup 

00 

l 

| W | |eTrine_e1Tin*|  > e) 

| e-4>  | <6 

n=-« 

n 1 

Let  nQ  be  chosen  so  that 


This  is  possible  since  J 

n=l 

Inequality  (3.9)  implies  that 


by  Lemma  3 


of  the  appendix. 


P(  sup  |WT(e)-WTU)|  > e) 

|e-4>|<6  T T 


< P(  sup  l |W  |n|e-*|  > ^ ) + P(  J |W  | > | ) 
I e-<t>  | <6  n=-nQ  |n|>n0 

nQ 

i 2.  P<l“nl  > Sb  ) * 2jnt , P<|W„I  >c„) 

n=l  0 n=nft+l 


(Here  en  = ~prS  ur!/3(  l . Mn/3)_1) 

c n=riQ+i 


2Trnn6  2 n0  °°  m_ 

< ( — 0_)  V y + y " 

- v e ; n nt-  .1  2 

n-1  n-nQ+l  en 


(since  E(Wn)  = 0 and  Var(Wn)  = un) 


(2Trnn6)2  n0  o 

— — i.  un  ♦ 4 a p.1/3) 


e n=l  " e n=n0+l 


n 


8 /e»3 


■ Cn„  T * T <t>' 


where  C depends  only  on  n . Choosing  6 such  that 


C — k <£  1.6.  0 < 6 < 

no  7 


K) 


we  have  the  inequality 

P(  sup  |WT(e)-WTU)|  > e)  <2e 
| e-(f>  | <6 

3 i i 


for  every  0 < 6 < ) * an<*  ^or  every  T > 0.  This  proves  (3.8) 

Theorem  3.1 . The  family  of  stochastic  processes  { Zj ( e ) , 9€  ©}  on 
C[-l.l]  converge  in  distribution  to  the  Gaussian  process  with  mean 
zero  and  covariance  function 


■MMM 


R(e1 ,e2)  = E[v(e1,X(0))v(e2,X(0))]o< 


4.  Strong  consistency 

Let  us  now  consider  the  limiting  process  RT(e)  defined  by  (2.9) 
Any  estimator  which  minimizes 

(4.1)  Me)  = /{f(e,X(t))-f(en,X(t))}zdt 

I Q U 


-2  /[f(0,X(t))-f(e  ,X(t))]dc(t) 

0 U 

is  called  a process  least  squares  estimator  of  0. 

Let  u be  the  measure  generated  by  the  process  X on  C[0,T]  when 

o is  the  true  parameter.  From  the  general  theory  of  diffusion  processes, 

the  Radon-Nikodym  derivative  of  y with  respect  to  y.  exists  and  is 

0 Oq 

given  by 


(4'2)  d = exp|/{f(e,X(t))-f(0  X(t))}dc(t) 

0O  '0 

l T 9 » 

- ? f {f(e,x(t))-f(e0,X(t))rdt}  . 

(cf.  Gikhman  and  Skorokhod  (1972),  p.90).  Hence 

109  a Sr  ■ - ? rt(9) 

eo 

which  proves  that  the  process  least  squares  estimator  eT  is  the  same  as 
the  maximum  likelihood  estimator  ©T  of  e(cf.  Basawa  and  Prakasa  Rao  (1979)) 
when  the  process  X is  observed  over  [0,T]. 
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Let 

T 2 

(4.3)  l,(e)  H / Cf(e,x(t))-f(en.x(t))3^dt 

1 0 u 

and  W*  be  a standard  Wiener  process.  Since  the  solution  of  the  stochastic 
differential  equation  given  in  Section  2 is  stationary  ergodic  by  hypothesis, 
it  follows  that  IT(e)  » a.s.  for  9 f 0Q  by  (A5)  and  the  process  {RT(o)> 

can  be  identified  with  the  process  { IT( e ) + 2W*(Tj(e))} . Furthermore 

(4.4)  IT(e)  + 2W*(TT(e))  --  a.s. 

as  T -*•  °°  for  any  e = eQ.  Hence  0 and  eQ  are  pairwise  consistent.  Note 
that 

(4.5)  RT(e)  = IT(o)  + /f  ZT(e),  see,  T>0 

where  IT(e)  is  defined  by  (4.3)  and  ZT(e)  is  given  by  (3.1).  Let 

(4.6)  Z*(e)  = /f  ZT(e). 

Then 

(4.7)  1 K0)  a-s-  as  T -v  » by  the  ergodic  theorem. 

In  order  to  study  the  strong  consistency  of  the  estimator  0j,  we  shall 
first  obtain  bounds  on  the  modulus  of  coninuity  of  IT(e)  and  Zf(e). 

Lemma  4.1 . Under  the  assumptions  (A1)-(A5), 

T 

|IT(0)-ITU)|  1 c,  |e-4>|  / J(X(t))(l+|X(t)|)dt  a.s. 

o 

where  is  a constant  independent  of  T,  0 and  $. 

Proof.  Note  that 

T 

IT(e)-ITU)  = /{f(X(t),o)-f(X(t),#)>- 
1 ' 0 

•{f(X(t),*)+f(X(t),o)-2f(X(t),o0)|dt 


and  therefore 


T 

1 1 T(o)-ITU)|  < |e-*|  / j(x(t))-{L(e)+LU)+2L(en)Hl+|X(t)|}dt 

0 u 

T 

< C , 1 6-4. | / J(X(t)  0+|  X(t)  | }dt  . 

0 

Remark.  Since  E[J2(X(0)]  < «=  and  E[X2(0)]  < It  follows  that 
E[J(X(0))X(0)]  < «»  and  hence  by  the  ergodic  theorem 
, T 

T / J(X(t)){l  + |X(t)|}dt  ^ E[J(X(0))0+|X(0)|>]  < - as  T 
0 

Therefore 

(4. 8)  |IT(e)-ITU)|  < C*T| e-4>|  a.s. 

as  T -+  » for  some  constant  C*  > 0.  In  view  of  (4.7)  and  Lemma  4.1, 
it  follows  that 

Me)  a , - 

(4.9)  -y-  ^ i(e)  = E[f(e,x(0)).f(e0,x(0))]2 

1lB  IT(0) 

uniformly  in  e e © as  T + ®.  But  I_(eJ  = 0 and  ^ — >0  a.s 

for  o f 0Q  by  (A5) . Hence,  for  any  6 > 0, 

, M0)  a s 

(4.10)  inf  ^4-  a as  T -*•  » 

I e-°oL>6 

for  some  A > 0 depending  on  6. 

Lemma  4.2.  Under  the  assumptions  (A1)-(A4),  for  any  Tq  > 0 and  any 

r:  > 0, 

Tn 

(4.11)  P(sup  sup  |Z*(e)|  > e)  < C,  -y 

9 0<T<J0  T “ 2 e 

for  some  constant  > 0. 
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Proof.  Let  h(e,x)  and  g(0,x)  be  defined  as  in  Section  3 and 
h(e,x)  = l an(x)e7rine,  e €[-1.1]. 


Let 

T 

W*  = / a (x(t) )dc(t ) . 
n o 

Since  g(e,x)  is  a cubic  polynomial  in  e with  coefficients  in  x,  it  is 
easy  to  check,  by  Kolmogorov's  inequality,  that 

T i 

(4.12)  sup  sup  | / g(e,X(t))dc(t) | = 0 (T*) 

6 °£t1t0  0 P 

using  the  fact  that  |e|  <_  1 . On  the  other  hand,  for  any  e > 0, 

(4.13)  P($up  sup  | l {/  a (X(t))dc(t)}e1Tl0(  > e) 

9 0<T<T0  n 0 n 

T 

< P(  sup  l | / a (X(t))dc(t)(  > e) 

0<T<T0  n 0 " 

T 

£ I P(  sup  |/  a (X(t ) )dc( t) | > e ) 
n 0<T<T0  0 n n 

(where  len<_e) 

1 To 

i l 4 Var {/  a (X(t))dc(t)) 

n e_  0 

n 

(by  Kolmogorov's  Inequality  for  martingales) 

1 T°  2 
ll  -J  f E(an(X(t)))Zdt 

n en  0 


(lu 


1/3,3 
n ' 
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when  cn  is  chosen  to  be  em^3  ([  Note  that  M = Eu^3  < «. 

n 

Hence  relations  (4.12)  and  (4.13)  together  prove  that 

P(sup  sup  |Z*(e)|  > e)  < C.  -S-  . 

0 0<T<T0  ' ~ 2 e2 

for  some  constant  C2  > 0 independent  of  Tp  and  e. 

Lemma  4.3.  For  any  y > 1/2,  there  exists  H > 0 such  that 


(4.14) 


lim  sup  sup  -T7> 
T -*■  co  e t1/z 


zx0)l 


< H a.s, 

(log  T)Y 


Proof.  Let 


= [ sup  sup  |Z,(e)|  > H'  2n/2nY],  n > 1. 
„n-l  T,0n  9 


2"  <T<2 

Observe  that  Lemma  4.2  gives  the  inequality 


) = P[  sup  sup  |ZT(0)|  > H'2n/2nY] 


0<T<2 


(by  stationarity  of  the  process  X(t)) 


H,22nY  2H*‘ 


Hence  £ P(A  ) < » which  implies  that  P(An  occurs  infinitely  often)  = 0 
n*l  " 

by  Borel -Cantel 1 i Lemma.  Therefore  sup  |ZT(e)|<  H'  2n/2nY  for  all 
n 1 6 l - 

2 < T <_2  except  for  finitely  many  n with  probability  one  and  hence 

lim  sup  sup  | ZT ( 0 ) | < H T1/2(log  T)Y  a.s. 

T ^ oo  0 

for  suitable  H > 0 depending  on  y. 

Theorem  4.1 . Under  the  assumptions  (A1)-(A5), 

0j  •>  0q  a.s.  as  T ® . 
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Proof.  Note  that 

RT(e)  = IT(e)  + Z*(o) 

and  RT(e0)  = 0.  Furthermore,  for  any  6 > 0,  there  exists  A > 0 depend- 
ing on  6 such  that 

inf  I,(e)  >.  Ta  a.s.  as  T 

|9-e0|>6 

by  (4.10)  and  with  probability  one,  for  any  y > p there  exists  H > 0 

depending  on  y such  that 

sup  | Zi(e) | < H T1/2(log  T)Y  a.s. 
e 1 

for  sufficiently  large  T.  Hence 

inf  R-r(e)  >_  a*T  > 0 a.s.  as  T + ®. 

|9-e0|>6 

for  some  A*  > 0 depending  on  6 and  y.  Since  eT  minimizes  R^fo)  and 
Rt(90)  = °»  **  Allows  that  1 0y— 0g | £ 6 a.s.  as  T ->  °°.  Hence  -»  eQ  a.s. 
as  T *►  °°  . 

5.  Asymptotic  normality  of  the  estimator 

In  addition  to  the  conditions  (A1)-(A5)  assumed  in  Section  2,  let  us 

suppose  that  there  exists  a neighbourhood  V.  of  en  such  that 

0O  u 

(A6)  |f^(e,x)|  < M(e)(l+|x|),  eev 

6 0Q 

and 

sup  (M(e) : eev.  } = M < <*>. 

°0 

We  shall  now  obtain  the  asymptotic  distribution  of  under  the  con- 

A A 

ditions  (A1)-(A6).  Since  0y  is  strongly  consistent,  e-j.  fc  V0  with  prob- 
ability one  for  large  T.  Expanding  f(e,x)  in  a neighbourhood  of  0g,  we 
have 


'I 
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f(e,x)  * f(e0.x)  + (0-eo)f(e,x) 
where  |o-o0|  < j e-e0 1 and  hence 

T _ 

(5-1)  IT(e)  5 I (f(0,X(t))-f(en,X(t))}2dt 

1 0 0 

s (9-0o)2  / {fJ1}(0o,X(t))}2dt 

+ (0-eo)2  f )(e,x(t))}2-{f^1)(e0.x(t))}23dt. 

Observe  that 

<5-2)  Uf<1)(0,x)}2-{f(1)(9o,x)}2| 

= l^1)(0,x)-f(1)(eo,x)nf(1)(e,x)+^1)Ceo.x){ 

< 2 M|e-e0|a  c(x)(l+|x| ) 
by  assumptions  (A3)  and  (A6).  Therefore 

(5.3)  |IT(e)  - (0-0Q)2  } {f0(1)(0o,X(t))}2dt| 


< 2 M|e-e0|^+a  / c(X(t))(l+|X(t)| )dt. 


Let  us  write  e-eQ  = T-1/>.  Then  it  follows  that 

(5'4)  U|<A  |lT{0)^2T"1/{f91)(0O*X(t)^}2dtl  iNiA^r1"1 


for  some  constant  Mj  > 0 by  the  erogodlc  theorem  since 

E(c(x(0))(l+|X(0)| ) < 

On  the  other  hand,  let 

vTU,x)  = T1/2[f(0o^r1/2,x)-f(eo,x)-<,T-V2f^)(eo,x)] 
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for  1 4/ 1 £ Ay.  Then  Vy(tf»,x)  is  differentiable  with  respect  to  v and 


the  derivative  vj1  ^U.x)  satisfies 


v^U.xJ-v^U.x)  = fJ1)(0o-hl-r1/2,x)-f^1)(eonT'1/2,x) 


and  hence 
(5.5) 


|vj1)(^,x)-vj.1)(c,x)|  < c(x)ra/2|*-c|a 


by  (A3)  for  all  ip , ^in  [-Ay, Ay].  It  can  be  shown  that  there  exists  a 
polynomial  in  i p with  coefficients  in  x viz 

(5.6)  gT(ip,x)  = Vy(Ay,x)P!(^-)  + Ayvj.1  ^(Ay,x)P2(^-) 

+ Vy(-Ay,X)P3(^-)  + Ayv|  ^(-Ay,X)P4(^-) 
on  [-Ay, Ay]  such  that 

(5.7)  9y(Ay,x)  = Vy( Ay ,x ) ,gy (“Ay ,x ) - Vy(— Ay,x) , 

(5.8)  g^(Ay.x)  = v^(Ay,x)  and  g^(-Ay.x)  = vj^(-Ay,x) 

where  P^ , 1 < i < 4 are  polynomials  in  with  constant  coeffiecients. 

Ill  * 

Observing  that  Vy(0,x)  = v|  '(0,x)  * 0,  it  is  easy  to  check  that 


(5.9) 


(5.10) 


(5.11) 


Ig^fAy.x)!  < c(x)AyT_a/2  , 

Ig^t-Ay.x)!  < c(x)A“ra/2  , 


|gy(Ay,x)|  < C(x)Ay  aT 


l+aT-a/2 


(5.12) 


|gT(-AT,x)|  < c(x)A|+aT'a/Z 


Furthermore  there  exists  a constant  > 0 Independent  of  T such  that 
(5.13)  Igy1  )(<k.x)-gy1  }(c,x)|  < M2c(x)A“-1T‘a/2|*-c| 
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for  all  t[-AT,AT].  But 

since  < 2A  . Hence  there  exists  a constant  M,  > 0 Independent  of 

T such  that 

(5.14)  |gj1,(*.x)-gj1,(c.x)|  < H3c(x)T“o/2U-c|01 
for  all  4 1,  ?€[-Aj,Ay] . Renormalizing,  we  get  that 

(5.15)  |gj1V,x)-gj1)(c*,x)|  < M3c(x)A“|4.*-c*!aT"a/2 
for  all  4)*,c*  € [-1 ,1  ].  Let 

(5.16)  hT4*,x)  = vT(i^*,x)-gT(4(*,x). 

Then  there  exists  a constant  > 0 independent  of  T such  that 

(5.17)  |h<1)(**,x)-h{1,(c*.x)|  < M*c(x)A“U*-c*|aT'a/2 

for  all  \i>*, /;*  6 [-1,1]  by  relations  (5.5)  and  (5.15).  Now,  applying 
Fourier  series  methods  as  in  Lemma  4.2,  it  can  be  shown  that  for  every 

c > 0, 

P(  sup  | / vT(**,X(t))dc(t)|  > c)  < A2oraE[c2(X(0)] 

|**|£l  01  “ e2  T 

and  hence 

(5.18)  P(  sup  | /{f(e0+*r1/2,X(t))-f(e0,X(t)) 

|4/|<At  0 

- ♦T"1/2fJ1)(e0,X(t))}dc(t)|  > e) 

A2oT”°E[c2(X(0)]  . 

e 

Let  us  choose  Ay  = log  T.  Since 


tSSESTT* 


l 

I 


f 

1 

♦ 

; 

' 
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T ] {fe1)(0O’x(t))}2dt  " 1{e0}  = E[f01)(Vx(o))]2  a*s' 

as  T ->•  oo  by  the  ergodic  theorem  and 

— / fJ1)(en,X(t))dc(t)  N(O,o2I(0o))  as  T -*•  “ 
/TO90  0 

by  the  central  limit  theorem  for  stochastic  integrals  (cf.  Basawa  and 

Prakasa  Rao  (1979)),  relations(5.4)  and  (5.18)  imply  that  the  asymptotic 

distribution  of  6y  which  minimizes  Ry(e)  given  by  (2.9)  can  be  obtained 

from  the  process 

(5.19)  I ( 6 q ) - 2i|iZ,  -°o  < ip  < oo 

2 

where  Z is  normal  with  mean  0 and  variance  o I(©q).  Since 

* « Z/I(e0) 

minimizes  (5.16),  it  follows  that 

(5.20)  Tl/Z(er0o)  N(O,o2/I(0o)). 

This  result  is  obtained  under  stronger  conditions  in  Prakasa  Rao 

(1979b)  for  the  least  squares  estimator  e T defined  at  the  beginning 

n , 1 

of  Section  2. 


Appendix 

Lemma  1 Suppose  <j>(u)  is  square  integrable  on  [-1.1]  and  <(.(•)  is 
Llpacnitz  of  order  a i.e.,  then  exists  c > 0 such  that 

(1)  |*(u)-*(v)|  < c|u-v|a  . 

Let  <p(u ) * l a e1^1™.  Then  for  any  0 < y < a. 
n n 

(2)  £|aJ2n2Y  I K-j  (o,y )c2  . 


I 


i 


; 


i 


i 
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Proof.  It  is  easy  to  check  that 

(3)  / 1 4>(u+h)-(|»(u-h)| ^du  = 4 £ | an | ^ sin2irnh. 

■1  n 

Since  <p  is  Lipschitz  satisfying  (1),  it  follows  that 

(4)  4 l |aj2  sin27rnh  < 22a+1c2h2a 

n 

for  all  h € [0,1].  Let  h = 2~k  and  2k”2  < n _<  2k~^ . It  is  clear  that 
2 1 

sin  nnh  >_  j and  relation  (4)  shows  that 

2k-l 

(5)  l | a | 2 < 22ac22"2ka 

k _?  n ~ 
n=2*  *+1 

for  any  k > 2 and  hence  for  any  0 < y < a, 

(6)  l | a | 2o2y  < 22»c22<2*-2«)k  _ 

n«2k'2*l 

Summing  over  all  k >_  2,  we  obtain  that 

(7)  l |a  |2n2y  < 22ac2(l-2(2Y-2a))-1  . 
n 

Hence  there  exists  a constant  ^(a.y)  > 0 such  that 

(8)  l |an|2n2y  ^ K^U.yJc2 

n 

where  c is  the  Lipschitzian  constant  given  by  (1). 

Remark.  A slight  variation  of  the  above  result  is  due  to  Szasz  (1922). 
The  proof  given  above  is  the  same  as  in  Szasz  (1922)  and  is  given  here 
for  completeness. 

Lemma  2.  Suppose  h(u)  is  square  integrable  on  [-1,1]  with  h(-l)*h(l)=0 
and  h'(*)  exists  and  is  Lipschitzian  of  order  o i.e.,  there  exists 
c > 0 such  that 
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(9)  | h*  (u)-h' (v)  | £c|u-v|a. 

Let  h(u)  = l ane1Tinu.  Then,  for  any  0 < y < a, 
n 

(10)  l |al2n2+2Y  < K?(a,y )c2 


(11)  l |aj2/3  < K,(a,y)c2  . 

n 

Proof.  Since  h'(u)  = ni  £ nane1Tlnu  , inequality  (10)  follows  from 

n 

Lemma  1 . Observe  that 

l <11^,1  V«V'3<In-"^)2/3 

n 

<K2(„,,)c2<In-<^!)2'3 
= K3(a,y)c2  . 

Lemma  3.  Let  h(e,x)  = £ an(x)e1T’n9  and  suppose  there  exists  a > 0 

n 

such  that 

|h^(e,x)-h^U,x)|  i c(x)|e-<t>|a 

for  all  e,  <|>in  [-1,1]  where  f^denotes  the  partial  derivative  of  f with 

0 

respect  to  e.  Let  (X(t),  t€[0,T]}  be  a stochastic  process  such  that 

E[h(e,X(t)]2  < - 

for  every  t €[0,T].  Then,  for  any  y < a,  there  exists  a positive  constant 
K4(o,y)  such  that 

l 4 / E[a2(X(t))]dt}1/3  < K4(a,y)  { l } E(c2(X(t))dt)1/3. 
n 0 n 1 0 


■MMMHMMfli 


I 


Proof.  By  Lemma  2,  it  follows  that 

l |an(X(t))|2n2+2y  < K2(a,Y)c2(X(t))  a.s. 
n c 

for  every  t€[0,T],  Hence 

I E[a2(X(t))]n2+2y  < K2(a,Y)E[c2(X(t))] 
for  all  t € [0,T] . Let 

1 T ? 

"n  = T / E[a‘(X(t))]dt. 

The  inequality  proved  above  gives  the  relation 

l %"2+2y  i M«.y)  T / E[c2(X(t))]dt 
n c ' 0 

and  hence 

z ,:/3  < 

n 1 

1 4/3(«.T)(Zn‘(1+Y))2/3  { i / E[c2(X{t))]dt}1/3 

1 0 

< K4(a,Y){  1/  E[c2(X(t))]dt>1/3  . 

1 0 

Remark.  Analgous  argument  proves  that 

b\'2  < ^m'l)yhb-z^)'12 

, 
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1.  Introduction 

Recently  there  is  a growing  interest  in  the  study  of  inference 
problems  for  stochastic  processes  both  continuous  and  discrete  time 
in  view  of  the  large  number  of  applications  to  engineering  problems. 

It  has  been  found  that  the  class  of  diffusion  processes  is  amenable 
for  statistical  analysis.  A survey  of  the  recent  work  in  this  area 
is  given  in  Basawa  and  Prakasa  Rao  (1979).  Further  work  on  asymptotic 
theory  of  maximum  likelihood  and  Bayes  estimators  for  parameters  of 
diffusion  processes  is  discussed  in  Prakasa  Rao  (1979a). 

Dorogovchev  (1976)  studied  weak  consistency  of  least  square 
estimators  for  parameters  of  diffusion  processes  which  are  solutions 
of  non-linear  stochastic  differential  equations.  Asymptotic  normality 
and  asymptotic  effiency  of  these  estimators  is  investigated  in  Prakasa 
Rao  (1979b).  Our  aim  in  this  paper  is  to  study  limiting  properties  of 
a process  related  to  least  squares  estimator  and  hence  to  discuss  the 
asymptotic  properties  of  an  estimator  derived  from  the  limiting  process. 
We  study  strong  consistency  and  asymptotic  normality  of  this  estimator. 
Our  approach  here  is  entirely  different  from  that  of  Dorogovchev  (1976) 
and  Prakasa  Rao  (1979b).  We  believe  that  our  techniques  for  study  of 
families  of  stochastic  integrals  is  new  and  is  of  independent  interest. 


c. 


i 


i 


B 


■ 


i 

■ 


2.  Study  of  process  related  to  least  squares  estimator 


Let  (X(t),t  ^ 0)  be  a real-valued  stationary  ergodic  process 
satisfying  the  stochastic  differential  equation 

dX(t)  = f(e0,X(t))dt  + d?(t),  X(0)  - X0,  t > 0 

2 2 

where  c( t ) is  a Wiener  process  with  mean  zero  and  variance  o t,  a 
being  unknown  and  E[XQ]  < ».  Suppose  f(e,x)  is  a known  real-valued 
function  continuous  on  ex  R where  e is  a closed  Interval  on  the  real 
line  and  9q  € 6 is  unknown.  Without  loss  of  generality,  assume  that 

0 = C-1,1]. 


Suppose  the  process  (X(t),  0 <_  t <_  T}  is  observed  at  time  points 
tR,  k = 0,1,..., n-1  with  tQ  = 0 and  tp  * T.  Let 

T "i1  WVn)  - Wk)  “ f(®,X(tk))At.  ]2 

Q>)  = l 5+1 5- . 

n k=0  \ 

where  AtR  = 0 1 k £ n-1 . An  estimator  0n  t which  minimizes 

Q^(n)  over  0€6is  called  a least  squares  estimator  of  e.  Assume  that 

* T 

such  an  estimator  exists.  Note  that  if  0 T minimizes  Q (0),  then  it 

n,T  xn 

minimizes  qj(e)  - oJ(e0). 

We  shall  first  study  the  limiting  properties  of  the  process 

IqI(o)  - Ql(0n)»  0€©}as  the  norm  of  division  a = max  It.  .,-t.  I tends 

n llkin  k+i  k 

to  zero.  Let  aXr  = X(tk+1 ) - X(tk)  and  A^  = C(tk+1)  - ?(tR),0  < k < n-1 . 
For  any  fixed  e. 


1 


I 


this  section  are  satisfied.  Since  f(e,x)  Is  continuous  In  x and  the 


4 


process  X has  continuous  sample  paths  with  probability  one.  It  follows 
that 

(2.1)  Iln  a^'  / [f(e0,X(t))-f(e,X(t))]2dt 

as  An  -*■  0.  Assumption  (A2)  implies  that 

(2.2)  I2n  / [f(e0,X(t))-f(e,x(t))]d?(t) 

as  An  -*■  0 in  view  of  stationarity  of  the  process  X where  the  last  in- 
tegral is  the  Ito-stochastic  integral. 

Let  us  now  estimate  I3n.  In  view  of  assumption  (A4),  it  can  be 
checked  that 
fck+l 

(2.3)  |/  {f(eo,X(t))-f(0o,X(tk))}dt| 

tk+l 

<L(e0)/  ]X(t)-X(tk))dt 
tk 

tk+l  t 

< L(e0)/  (U(t)-c(tk)|+/  f(e0,X(s))|ds}dt 

*k  tk 

tk+l  9 tk+l  t 

< l(eQ)/  U(t)-c(tk)|dt+L  (0Q)/  [ /{l+|X(s)|}ds]  dt 

t * t t 

h rk  \ 

< L(e0)Atk  sup  |?(t)-5(tk)|+L  (eQ)Atk  sup  / (1+| X(s) j }ds. 

tk-t-tk+l  tk-t-tk+l  tk 

< L(en)At.  sup  U(t)-c(tk)|+L2(e0)Atk  sup  {1+| X(t) | } 

- k+1  - k+1 

for  0 £ k _<  n-1 . Using  assumption  (A4)  again,  we  obtain  the  following 
inequal ity: 
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<2-4)  I3n  £ I J(X(tk)){L(e0).Atk  sup  U(t)-c(tk)| 
k tk-t-tk+l 

+ l2(9o)Atk  SUP  {l+|X(t)|})|e-e0|. 
tk^tk+1  f 

Since  J( • ) is  continuous  and  X(»)  has  continuous  sample  paths  al- 
most surely,  it  follows  that  there  exists  a constant  C*( oQ)  depending 
on  T only  such  that 

(2.5)  I3n  £ c*(e0){lAtk.  sup  |c(t)-c(tk)|  + [At2}|e-e0|  . 

(k  tk-t-tk+l  k 9 

Since  6€S  compact,  it  follows  that 


whenever  Afl  is  sufficiently  small  by  the  law  of  iterated  logarithm  for 
Brownian  increments  (cf.  McKean  (1969),  p.14).  Therefore 

(2-6)  I3n  = 0([  Atj|/2  log  1/2  1/Atk)  a.s. 

uniformly  in  efc  © . Furthermore  the  convergence  in  (2.1)  is  uniform  in 
e € © since 

|f(0o,X(t))-f(e,X(t))|2  < |eo-0|2j2(X(t))  < C J2(X(t)) 

and  J(X(t) ) is  integrable  pathwise  on  [0,T]  by  (A4).  Here  we  have  used 
the  fact  that  © is  compact.  Hence 

T , 

(2.7)  !in  " / C^(0o»X(t)-f(0 ,X(t)]  at  + o(l)  a.s. 

uniformly  in  e as  An  ■+  0.  We  shall  discuss  uniform  convergence  of  Ig 
in  the  next  section. 

Relations  (2.0),  (2.6)  and  (2.7)  show  that,  for  any  fixed  T, 
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(2.8)  Qj(0)-Qj(0o)  = / [f(0o,X{t))-f(e,X(t))]2dt  + I^+oO)  «-S. 


uniformly  in  9£  <b  compact  as  An  -+•  0 where  l^n  satisfies  relation  (2.2). 
Let  us  consider  the  limiting  process 


(2.9)  R (e)  = / [f(en,X(t))  - f(e,X(t))]2dt 
1 0 u 

T 

+ 2 ^ Cf(e0,x(t))  - f(o,x(t))]dc(t) 


= / v2(e,X(t))dt  - 2 / V(e,x(t))dc(t) 
o 0 

where 

(2.10)  v(e,x)  = f(e,x)  - f(e0,x). 


We  study  the  limiting  properties  of  the  process  {R-[-(q),  e€0}  In 
the  next  section. 


Assumptions 

(A! ) f(e,x)  is  continuous  in  (e,x)  and  differentiable  with  respect  to 
o.  Denote  the  first  partial  derivative  of  f with  respect  to  e by 
f^(o.x)  and  the  derivative  evaluated  at  9q  by  f^(0Q,x). 

(A2)  E[fJ1)(90,X(0))]2  < co 

(A3)  f^(9,x)  is  Lipschitzian  in  9 for  each  x i.e.,  there  exists  a > 0 

such  that 

Ifg1  ^(e.x)-fg1  ^U,x)|  < c(x) | e-4>| 01  » xtR.e.fte 
and 

E [c2 ( X ( 0 ) ) ] < 

(A4)  f(o,x)  satisfies  the  following  conditions: 
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(i)  | f ( e ,x ) } <.L(e)(l+|x|)  , 0 © © , xf  R ; sup{L(e):  0©  ©)  < . 

( 1 i ) | f(9,x)-f(8,y)|  < L(e ) | x-y | , 0€  © , x,y,  t R. 

(iii)  | f(e»x)-f(4*,x) | <_  j(x)|e-$| , e , <t>©  © , x © R 

p 

where  J(*)  is  continuous  and  E[J  (X(0))]  < °°. 

(A5)  1(6)  h E[f(0,X(O))-f(0o,X(O)]2  >0  for  0 / 0Q. 

2 

Remark:  Since  E[X  (0)]  < assumption  A4(i)  implies  that 

E[f(0,X(O))]2  < » 

for  all  0€  G . 

3.  Study  of  a limiting  process  related  to  least  squares  estimator 
Let  us  now  study  the  properties  of  the  limiting  process 

T T 

(3.1)  Z,(0)=-L  / v(a*X(t))dt(t) 

' /f  o 

as  a process  in  the  parameter  e©  0 = [-1,1]  as  T -*•  «.  From  the  central 
limit  theorem  for  stochastic  integrals  (cf.  Basawa  and  Prakasa  Rao  (1979)), 
it  can  be  shown  that 

~ / v(0,X(t))d£(t)  -£•  N(O,E[v(0,X(O))]2o2) 

/r  o 

since  the  process  X is  stationary  ergodic.  In  general,  finite  dimension- 
al distributions  of  the  process  { Zy( e ) , 0©  © } converge  to  the  finite 
dimensional  distributions  of  the  Gaussian  process  (Z(o),  e © ©}  with  mean 
zero  and  covariance  function 

R(ere2)  = E[v(0rx(O))v(e2,X(O))]a2. 

We  shall  now  prove  the  weak  convergence  of  the  process  (Z^e),  e©6 } 
on  C[-l,l]  under  uniform  norm.  It  is  sufficient  to  prove  that 

(3.2)  lim  Tim  P(  sup  | ZT(o)-ZT(<j>)|  > c)  = 0. 

T-*»  6->0  1 0 — <)>  | ^ 6 
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Since  v(e,x)  is  differentiable  with  respect  to  e on  [-1,1]  by 
assumption  (Al),it  is  easy  to  see  that  there  exists  a cubic  polynomial 


g(o,x)  in  0 such  that 


g(-l,x)  = v(-l,x).  g(l,x)  = v(l,x) 


g^H.x)  = v^H.x).  g^d.x)  = v^O.x). 


h(e,x)  = v(e,x)-g(e,x). 

Then  h(-l,x)  = h(l,x)  = O.hJH'-l.x)  =hp\l,x)  = 0.  Now 


1 T , T 

(3.3)  ZT(e ) = — / h(e,x(t))dc(t)  + — / g(0,X(t))dc(t). 
/TO  /TO 


Since  g(e,x)  is  a cubic  polynomial  in  e with  coefficients  in  x which  are 
linear  functions  of  v(-l,x),  v(l  ,x)  ,v^(-l  ,x)  and  v(0]  J(1  ,x) , it  is  easy 
to  check  the  uniform  equi-continuity  condition  of  type  (3.2)  for 


i T 

— : / g(e,x(t))dc(t). 
/f  0 


Let  us  now  consider  the  process 


(3.4) 


WT(0)  = 4;  / h(0,X(t))dc(t). 
1 A 0 


Let  the  Fourier  expansion  for  h(0,x)  in  L2([-l,l])  be  given  by 


(3.5) 


h(o,x)  = l an(x)e* 
n 


Lemma  3.1 


T T < a 

(3.6)  / h(0,X(t))dc(t)  = H J an(X(t))d?(t)}e7r1ne 

0 n 0 n 


in  the  sense  of  convergence  in  quadratic  mean. 
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Proof  An  approximating  sum  in  L^-norm  for 


/ h(e,X(t))d£(t) 
0 


is 


a,n  - l 

J * 


and  an  approximating  sum  in  L9-norm  for  £{  / a (X(t))dc(t)}en1n0  is 

c n 0 


l e1Tln9(  l *„(X(t , ,)AC J • 
n| <M  j=l  n 3 3 


N 


2NM 


It  is  sufficient  to  prove  that  ^lA^-A^^I  ■+  0 as  N ->  » and  M •+  °°.  Now 


irine.,  i2 


= E|  l I a (X(t,  1))einnV1 
j=l  |n|>N  n 3 1 3 


<[  I (E  (I  an(X(t.  1))ACi)2}i]2 
| n | >M  j=l  " 3 1 3 

by  the  elementary  inequality 

E|Z 

n n 

for  any  sequence  of  complex  numbers  { xn > and  any  sequence  of  real 
valued  random  variables  {Yn,  n >_  1 } . Hence 

^IN-'W2  i U < I e<-„(«‘m»S,1]2  • 

| n | >M  j=l  J J 


E(an(X(tj_1))2Atj  - / E(an(X(t)}2dt  = Tun  (say). 


Since 
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I 

i 


as  N > »,  It  is  sufficient  to  prove  that  [ u|  < ».  This  follows  from 

n 

remarks  following  Lemma  3 of  the  appendix  under  assumption  (A3). 

Let 

, T 

<3-7)  wn  = ~ / a (X(t))d€(t). 

/f  0 n 

Lemma  3.2.  For  every  e > 0, 

(3-8)  Tim  P(  sup  |WT(e)-WT(<(,)|  > e)  * 0 

6->0  | 0-<fr  I <6 

for  every  T > 0. 

Proof.  In  view  of  Lemma  3.1,  for  any  e > 0, 

(3.9)  P(  sup  |WT(e)-WTU)|  > e) 

( e-<f>  J <6 


! 


= p( 

sup 

| l K > t) 

n=-oo 

( e-4»  | <6 

< P( 

sup 

| e-<t>  | <6 

i |W  | > c) 

n=-® 

Let  n^  be  chosen  so  that 

(3.10)  l 1/3  , e2-4/3  # 

n=n0 

This  is  possible  since  £ u 73  < ® by  Lemma  3 of  the  appendix. 

n=l  n 

Inequality  (3.9)  implies  that 


1 


1 


J 
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P(  sup  |WT(e)-WTU)|  > e) 

| e-<j>|  <6  ' T 


<p(.  SUP  l I W |n|e-4>|  > ) + p(  y | W I > | ) 

1 0-0 1 <6  n=-nQ  " |nT>n0 


± \ P('Wni  >^)+2?n+1  P(|Wn'  >En) 

n=l  0 n nQ+ I 


(Here  en  = -prs  uJ/3(  ? +1  ^/3)-1} 

t n=riQ+l 

2Trnri5  ^ n0  “ u 


cirnno  u “>  u 

< ( — o_)  y y + y 4 

— e _£•<  n t , 2 

n-1  n-nQ+l  ^ 


(since  E(Wn)  = 0 and  Var(Wn)  = yn) 
(2wn06)2  n0  o ' 

£—  l »n*4  (I  ./3) 

e n=l  e n=n0+^  0 


- r 5 + 8 /e\3 

~ Cn_  T + T (2} 

U e e 


where  C depends  only  on  n . Choosing  6 such  that 


C„  -y  < e i.e.  o < 6 < 

n0  7 


we  have  the  inequality 


P(  sup  | wT(e )-WT(<i> ) | > c)  < 2c 

| 0-<j)  ] <6 


for  every  0 < 6 < l^rT")  * and  ^or  every  T > °-  This  proves  (3.8), 
Theorem  3.1 . The  family  of  stochastic  processes  { ZT ( e ) , ee  G } on 
CC-1,1]  converge  in  distribution  to  the  Gaussian  process  with  mean 


zero  and  covariance  function 
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I 


as  T 


R(91.02)  = E[v(e1,X(0))v(e2,X(0))]o5 


4.  Strong  consistency 


Let  us  now  consider  the  limiting  process  RT(e)  defined  by  (2.9). 

Any  estimator  e-j-  which  minimizes 

T . 

(4.1)  Rt(q)  e / {f(e,X(t))-f(en,X(t))}1 2dt 
i o u 

T 

-2  /[f(0,X(t))-f(0n,X(t))]d€(t) 

0 0 

is  called  a process  least  squares  estimator  of  0. 

Let  Vje  be  the  measure  generated  by  the  process  X on  C[0,T]  when 

o is  the  true  parameter.  From  the  general  theory  of  diffusion  processes, 

the  Radon-Nikodym  derivative  of  p with  respect  to  y exists  and  is 

60 

given  by 

(4'2)  XT'  = exp|/{f(0,X(t))-f(0  ,X(t))}dc(t) 

M0O  ' 0 u 

1 T O V 

- j/  {f(0,X(t))-f(0o,X(t))}ZdtJ  . 

(cf.  Gikhman  and  Skorokhod  (1972),  p.90).  Hence 

dP0  1 

109  sr  =-2rt<»> 


0 


which  proves  that  the  process  least  squares  estimator  ©T  is  the  same  as 
the  maximum  likelihood  estimator  0T  of  e(cf.  Basawa  and  Prakasa  Rao  (1979)) 
when  the  process  X is  observed  over  [0,T]. 


i 
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(4.3) 


Ir(e)  i / [f(e,x(t))-f(e0,X(t))rdt 
1 o 


and  W*  be  a standard  Wiener  process.  Since  the  solution  of  the  stochastic 
differential  equation  given  in  Section  2 is  stationary  ergodic  by  hypothesis, 
it  follows  that  Ij(e)  « a.s.  for  e f 0Q  by  (A5)  and  the  process  IRy(o)} 

can  be  identified  with  the  process  (Ij(e)  + 2W*( T^( 0 ) ) } . Furthermore 

(4.4)  IT(e)  + 2W*(TT(e))  -*•  » a.s. 

as  T + ■»  for  any  e = eQ.  Hence  e and  0g  are  pairwise  consistent.  Note 
that 

(4.5)  RT(e)  = lT(e)  + A ZT(e),  efe,  T>0 

where  IT( e ) is  defined  by  (4.3)  and  ZT(e)  is  given  by  (3.1).  Let 


(4.6) 


Z*(e)  = A Z-(e), 


(4.7)  y IT(0)  K0)  a.s.  as  T -*■  » by  the  ergodic  theorem. 

I n order  to  study  the  strong  consistency  of  the  estimator  e^.,  we  shall 
first  obtain  bounds  on  the  modulus  of  coninulty  of  Ij(e)  and  Zf(o). 

Lemma  4.1.  Under  the  assunptions  (A1)-(A5), 

T 

|IT(e)-ITU)|  <.  C1 1 e-4>|  / J(X(t))(l+|X(t)|)dt  a.s. 

where  is  a constant  independent  of  T,  0 and  4. 

Proof.  Note  that 


IT(e)-ITU)  ■ /(f(X(t),e)-f(X(t),$)}* 

1 ' 0 

•{f(X(t),o)+f(X(t),o)-2f(X(t),o0)ldt 
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and  therefore 

T 

|IT(o)-lTU)|  < | e-4> | / j(X(t))-{L(e)+L(  + )+2L(e0)Hl+|X(t)|}dt 

T 

< C,|e-((>|  / J(X(t){l+|X(t)|  }dt  . 

1 0 

Remark.  Since  E[J2(X(0)]  < °°  and  E[X2(0)]  < »,  it  follows  that 
E[J(X(0))X(0)]  < °°  and  hence  by  the  ergodic  theorem 
, T 

j / J(X(t)){l  + |X(t)|)dt  E[J(X(0)){1  + |X(0)|}]  < co  as  T -*■  «. 

0 

Therefore 

(4.8)  |IT(e)-IT(4)|  < C*T|e-4|  a.s. 

as  T + " for  some  constant  C*  > 0.  In  view  of  (4.7)  and  Lemma  4.1, 
it  follows  that 

Me)  a c ? 

(4.9)  ^ 1(e)  = E[f(e,x(0))-f(e0,x(0))]2 

Um  IT(e) 

uniformly  in  9 € © as  T + •.  But  IT(eQ)  = 0 and  ^ _L_ — > o a.s. 
for  o t e0  by  (A5).  Hence,  for  any  6 > 0, 

Us)  a , 

(4.10)  inf  x as  T + « 

I e-00l--6 

for  some  A > 0 depending  on  6. 

Lemma  4.2.  Under  the  assumptions  (A1)-(A4),  for  any  Tq  > 0 and  any 

c '•  0, 

Tn 

(4.11)  P(sup  sup  |Z*(e)|  > e)  < C,  -y 

9 0<T<T0  T Z E2 

for  some  constant  C,,  > 0. 
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Proof.  Let  h(e,x)  and  g(e,x)  be  defined  as  in  Section  3 and 


h(e.x)  = l an(x)e 
n 


irine 


etc-i.i]. 


Let 


W * ■ / a (x(t) )dc(t)  . 

n o 


Since  g(e,x)  is  a cubic  polynomial  in  e with  coefficients  in  x,  it  is 
easy  to  check,  by  Kolmogorov's  inequality,  that 


(4.12) 


T 

sup  sup  | / g(e,X(t))de(t) | = 0 (T*) 
6 0<T<T0  0 p u 


using  the  fact  that  |e|  _<  1 . On  the  other  hand,  for  any  e > 0, 


(4.13)  P( sup  sup  \ Hf  a (X(t))dc(t)}eiri9|  > e) 
9 0<T<T0  n 0 " 


< P(  sup  l | / a (X(t))d5(t)|  > e) 

A ,T  .T  _ A » ' 


0<J<_Tq  n 0 


< l p(  sup  1/  a (X(t) )dc(t) I > e ) 

M A.T.T  A '• 


n 0<T<T0  0 


(where  £e  < e) 


< l \ Var(/  a (X(t ) )dc(t) ) 
- -c  0 n 


n 


(by  Kolmogorov's  Inequality  for  martingales) 
< l 1°  E(an(X(t)))2dt 


n ^ 0 


= t y — 

'o  1 2 

n en 


1/3\3 


i W'r 

G 
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when  v is  chosen  to  be  euV3  (£  . Note  that  M s < ». 

ti  n n **  n 

n 

Hence  relations  (4.12)  and  (4.13)  together  prove  that 

T„ 


P(sup  sup  |z$(e)|  > e)  < C 0 


6 °lTiT0  ' T 


- l2  T 

c 


for  some  constant  C2  > 0 independent  of  TQ  and  e. 

Lemma  4.3.  For  any  y > 1/2,  there  exists  H > 0 such  that 


(4.14) 


lim  sup  sup  -j-p? 
T 9 yu 


zT(0)l 


(log  T) 


y - 


< H a.s. 


Proof.  Let 


A-  = C sup  sup  | ZT(e) | > H'  2n/2nY],  n > 1. 
->n-l „n  9 1 “ 


2"  <T<2 

Observe  that  Lemma  4.2  gives  the  inequality 

P(A  ) = P[  sup  sup  | ZT(e) | > H’2n/2nY] 

„.T.on-l  9 1 


0<T<2 


(by  stationarity  of  the  process  X(t)) 


< C 2"-1  = c 

~ H,*2nn2Y  2H*‘ 


1 

~2y 


Hence  £ P( A ) < ~ which  implies  that  P(An  occurs  infinitely  often)  = 0 

n*l  n 


by  Borel-Cantelli  Lemma.  Therefore  sup  | Z_(e) I <_  H * 2n/2nY  for  all 

0 * 


?n  1 < T _<  2n  except  for  finitely  many  n with  probability  one  and  hence 


lim  sup  sup  I ZT(e)|  < H T1/2(log  T)Y  a.s. 

T “►  OO  0 


for  suitable  H > 0 depending  on  y. 

Theorem  4.1 . Under  the  assumptions  (A1)-(A5), 


By  > eQ  a.s.  as  T -*■  « . 


J 
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; 

! 


i 

1 


' 


! 


Proof.  Note  that 

RT(e)  = IT(e)  + Z*(o) 

and  Rj(6q)  = 0.  Furthermore,  for  any  6 > 0,  there  exists  A > 0 depend- 
ing on  6 such  that 

Inf  1,(0)  >.  Tx  a.s.  as  T -»  » 

|9-0O|>6 

by  (4.10)  and  with  probability  one,  for  any  y > p there  exists  H > 0 

depending  on  y such  that 

sup  | Zi(e ) ( <^H  T1//2(log  T)Y  a.s. 

0 1 

for  sufficiently  large  T.  Hence 

inf  R,(e)  >.  A*T  > 0 a.s.  as  T •*■  ». 

|0-0OI>6 

for  some  A*  > 0 depending  on  6 and  y.  Since  eT  minimizes  RT(e ) and 
Rj(0q)  = 0,  it  follows  that  | eT-e0|  _<  6 a.s.  as  T ■+  ®.  Hence  eT  -»  eQ  a.s. 
as  T •*  <»>  . 


5.  Asymptotic  normality  of  the  estimator 

In  addition  to  the  conditions  (A1)-(A5)  assumed  in  Section  2,  let  us 

suppose  that  there  exists  a neighbourhood  VQ  of  en  such  that 

e0  0 

(A6)  |^1}(0,x)|  < M(0)(H|x|),  0€ V„ 

e - e0 

and 

sup  (M(e):  0€  VA  } ■ M < ». 

90 

We  shall  now  obtain  the  asymptotic  distribution  of  eT  under  the  con- 

A A 

ditlons  (A1)-(A6).  Since  0y  is  strongly  consistent,  0jtV0  with  prob- 
ability one  for  large  T.  Expanding  f (e ,x ) In  a neighbourhood  of  0Q,  we 
have 
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f(e,x)  3 f(0o»x)  + (0-eQ)f(e,x) 
where  | o-oQ|  < | e-eQ|  and  hence 

T 

f5-1)  M0)  = I {f(0,X(t))-f(e.,X(t))}Zdt 

1 o 0 

= (0-0O)2  / {fg1 )(0O»X(t))}2dt 
+ (e-0o)2  / CtfJ1)(e.X(t))}2-{fJ1)(e0.X(t))l2]dt. 

Observe  that 

<5-2)  |{fe1)(0.x)}2-{f^1)(0o,x)}2| 

* lfl1)(9'*x)-fe1)(eo*x)llfe1)(o»x)+fe1,(0o*x)l 

£2  M|e-eor  c(x)(H|x|) 
by  assumptions  (A3)  and  (A6).  Therefore 


(5.3)  I IT(e)  - (e-e0)2  / tfg1 )(0o»X(t)) }2dt| 


<2  M| 0-0  | 2+0  / c(X(t))(l+| X(t)| )dt. 

0 

Let  us  write  0-0Q  = T~1/2#.  Then  It  follows  that 

(5.4)  sup  |lT(0)-/T-1/{f^)(0  X(t))}2dt|  < M^^T'1-01 
( ip  [jfAj  0 — i i 

for  some  constant  > 0 by  the  erogodic  theorem  since 

E(c(x(0))(l+|X(0)| ) < co. 

On  the  other  hand,  let 

vT(*,x)  = T1/2[f(0o+*T"1/2ix)-f(0o,x)-^T"1/2fJ1,(eo,x)] 
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for  |i|/|  £ Ay.  Then  Vy(ij/,x)  is  differentiable  with  respect  to  ip  and 
the  derivative  vj^U.x)  satisfies 


vj^U.xJ-v^U.x)  = fJ1)(0o+*T"1/2,x)-fJ1)(eo+i;T'1/2,x) 


i 


and  hence 

(5.5)  |v|^(<»,x)-v^(c.x)|  < c(x)T“a/2|^|a 

by  (A3)  for  all  ip , ein  [-Ay, Ay].  It  can  be  shown  that  there  exists  a 
polynomial  in  ip  with  coefficients  in  x viz 

(5.6)  gTU,x)  = Vj(AT»x)pi  (^)  + Atv^(At.x)P2(^-) 

+ vt(-At,x)P3(^-)  + Atv^  )(-At,x)P4(^-) 
on  [-Ay, Ay]  such  that 

(5.7)  9y(AytX)  = Vy( Ay ,X ) ,gy ( ”Ay »X ) - Vy(-Ay,x), 


(5.8)  g^(Ay,x)  = v^(Ay,x)  and  g^(-Ay.x)  = v^(-Ay.x) 

where  P. , 1 < i < 4 are  polynomials  in  with  constant  coeffiecients. 

1 ~ Mj 

(1  ) 

Observing  that  Vy(0,x)  = Vy  /(0,x)  = 0,  it  is  easy  to  check  that 

(5.9)  Ig-y-1  }(Ay  ,x)  | <c(x)AyT"a/2  , 


(5.10)  Igy1  )(~Ay,x) | £ c(x)AyT"a/2  , 

(5.11)  |gT(AT,x)|  < c(x)A}+aT'a/2  , 


and 

(5.12)  (gy(-Ay.x) ( < c(x)Aj+aT'a/2  . 


Furthermore  there  exists  a constant  > 0 Independent  of  T such  that 
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for  all  fc[-AT,AT].  But 

since  | »p-c|  1 2A  . Hence  there  exists  a constant  M-  > 0 independent  of 
T J 

T such  that 

(5.14)  |g|1)(i»,x)-gj1)(c,x)|  < M3c(x)ro/2|1|;-?|a 
for  all  iii,  c€[-Aj,Ay].  Renormalizing,  we  get  that 

(5.15)  | gj1  )(i|;*»x)-g|1  ^(c*,x)|  < H3c(x)AaT|**.C*|ara/2 

for  all  € [-1 ,1].  Let 

(5.16)  hT(i^*,x)  = vT(i/i*,x)-gT(i/)*,x). 

Then  there  exists  a constant  M3  > 0 independent  of  T such  that 

(5.17)  jh^V.xJ-h^U'.x)!  <M*c(x)A“|**-C*|Ya/2 

for  all  iii*,/;*  €[-1,1]  by  relations  (5.5)  and  (5.15).  Now,  applying 
Fourier  series  methods  as  in  Lemma  4.2,  it  can  be  shown  that  for  every 

c > 0, 

P(  sup  1/  vTU*,X(t))dc(t)|  > e)  < -X-  A2oTaE[c2(X(0)] 

|**|<1  01  ~ e2  T 

and  hence 

(5.18)  P(  sup  | /{f(en^T'1/2,X(t))-f(en,X(t)) 

|*|<At  0 0 0 

- ♦T"1/ZfJ1)(0o.X(t))}de(t)|  > e) 
A2aT‘aE[c2(X(0)]  . 


Let  us  choose  A-j.  = log  T.  Since 
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( 


T l ^e1)(e0’X{t))}2dt  - I(0O)  = a-s- 

as  T -»  <»  by  the  ergodic  theorem  and 

— / fJ^e^XUndcU)  N(0,o2I(e  ))  as  T + ® 

/f  0 9 u 

by  the  central  limit  theorem  for  stochastic  integrals  (cf.  Basawa  and 
Prakasa  Rao  (1979)),  relations(5.4)  and  (5.18)  imply  that  the  asymptotic 
distribution  of  eT  which  minimizes  Ry(e)  given  by  (2.9)  can  be  obtained 
from  the  process 

(5.19)  ^2I(6q)  - 2\pZ , -oo  < ^ < a. 

2 

where  Z is  normal  with  mean  0 and  variance  a I (0q) - Since 

* = Z/I(e0) 

minimizes  (5.16),  it  follows  that 

(5.20)  T1/2(eT-eo)  N(O,a2/I(0o)). 

This  result  is  obtained  under  stronger  conditions  in  Prakasa  Rao 
(1979b)  for  the  least  squares  estimator  en  j defined  at  the  beginning 
of  Section  2. 


Appendix 

Lemma  1 Suppose  4>(u)  is  square  integrable  on  [-1,1]  and  $(•)  is 
Lipschitz  of  order  a l.e.,  then  exists  c > 0 such  that 

(1)  U(u)-<f>(v)|  < c|u-v|a  . 

Let  4>(u ) = l anewinu.  Then  for  any  0 < y < a, 
n 

(2)  I|an|2n2Y  < Ki (a,y )c2  . 
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Proof.  It  is  easy  to  check  that 

(3)  / (<f>(u+h)-<(>(u-h)|2du  = 4 £ (aj2  sin2unh. 

-1  n 

Since  $ is  Lipschitz  satisfying  (1),  it  follows  that 

(4)  4 £ |anl2  sin2irnh  < 22a+1c2h2a 

n 

for  all  h € [0,1].  Let  h = 2~ k and  2k‘2  < n < 2k"^ . It  is  clear  that 
2 1 

sin  wnh  and  relation  (4)  shows  that 


(5) 


f l»/  £ 22acV2lw 

n=2k-2+l 


for  any  k _>  2 and  hence  for  any  0 < y < o, 
,k-l 

(6)  1 

n=2k'2+l 

Summing  over  all  k 2,  we  obtain  that 


l laJ2"27  £ 22ac22(2Y-2a)k  . 


(7)  l |al2n2Y  < 22ac2(l-2(2Y-2a>)-1  . 
n n 

Hence  there  exists  a constant  K^(a,y)  > 0 such  that 

(8)  l |an|2n2y  £ (a,y)c2 

n 

where  c is  the  Lipschitzian  constant  given  by  (1). 

Remark . A slight  variation  of  the  above  result  is  due  to  Szasz  (1922). 
The  proof  given  above  is  the  same  as  in  Szasz  (1922)  and  is  given  here 
for  completeness. 

Lemma  2.  Suppose  h(u)  is  square  integrable  on  [-1,1]  with  h(-l)=h(l)=0 
and  h * ( - ) exists  and  is  Lipschitzian  of  order  o i.e.,  there  exists 


c > 0 such  that 
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. 

: 

i 

i 


I 


t 


(9) 

|h'(u)-h'(v)|  ic|u-v|a. 

Let  h(u)  = l ane*inu, 
n 

Then,  for  any  0 < y < a , 

(10) 

l [a„|2n2+2y  < K2(a.Y)c2 
n 

and 

(11) 

I |a  |2/3<  K3(a,Y)c2  . 
n 

Proof.  Since  h'(u)  = ni  J.  naneirinu  , inequality  (10)  follows  from 

n 

Lemma  1 . Observe  that 

I |a//3i(I|a/n2t2y)'/3(In-<Uy>>2/3 

n 

< Kj(a,v)c2an-(’+''>)2/3 

“ Kj(a,y)c2  . 

Lemma  3.  Let  h(e,x)  = l an(x)eirin9  and  suppose  there  exists  a > 0 

n 

such  that 

|h^(e,x)-h^U,x)|  < c(x) | e-^l01 

for  all  e,  4>  i n [-1,1]  where  f^J denotes  the  partial  derivative  of  f with 

0 

respect  to  e.  Let  (X(t),  t€[0,T]}  be  a stochastic  process  such  that 

E[h(e,X(t)]2  < - 

for  every  t €[0,T].  Then,  for  any  y < a,  there  exists  a positive  constant 
K4(a,y)  such  that 

l (i  / E[a2(X(t))]dt}1/3  < K4(a,Y)  { i / E(c2(X(t))dt}1/3. 
n 0 0 


MV  ■" 

n 
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T 


t 


Proof.  By  lemma  2,  it  follows  that 

I|an(X(t))|V+2Y  < K2(«.Y)c2(X(t))  a.s. 
n 

for  every  t€[0,T].  Hence 

l E[a2(X(t))]n2+2Y  < K2(a,y)E[c2(X(t))] 
n 

for  all  t € [0,T] . Let 

1 T ? 

Mn  • T / E[a2(X(t))]dt. 

The  inequality  proved  above  gives  the  relation 

l ^nn2+2y  1 Ma.y)  y / E[c2(x(t))]dt 
n n c ' 0 

and  hence 

i »:/3  < 

n 

1 4/3(a,Y)(In-(1+Y))2/3  { l / E[c2(X(t))]dt}1/3 

1 0 

£ Moc.yH  \ ) E[c2(X(t))]dt}1/3  . 

1 0 

Remark.  Analgous  argument  proves  that 

b\n  <_  av°™'l)V2iKw«))''2 
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